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DOCUMENT" I DENT IF I l^R: US 2 0 020064302 Al 

TITLE: Virtual coBmetic autosurqery via ;:eleraeclicine 



A •••^i:thcd and apparatus; ;:or per ilcrff-inq -vir-:-:;;^! cosai^tic Burg^ry by input;-::; ng a 
patient '8 image, personal clata^ and proposed image vr-odi float: ion scheme int:c one of a 
plnralit:y of reixote computers, v^;lth the image and the inodi float ion schevi:e being 
analysed and modified by a central computer, thereafter displaying a modified 
patient image at: the revuote ooiriputer, \^ii:h the central computer beinq connected to 
the pluralit:y of such remote oomputerB via t:he Internet. 

>su\virc:ary of Invention Paragraph (9) : 

[0008] To facilitate the educatioxi process of a person interested in cosmeuio 
plastic surgery, several cowipanles have introd-iced computer softv^are proc>ravv:s which 
have the capability of niodifylnq the appearance of a prospective patient . These 
sliviulation and morphing soft:ware proaran^s are comraercially targeted for offices of 
surgeons perf orvnmg cosvrietic and plastic procedures, and constitute a useful tool in 
helping patients to better understand a partic-ilar cosmetic procedure and the effect 
of that: procedure on the patient ' s final postoperative appearance, should surgery be 
elected by the patient. 

Suvi-n-ary ot Invention Paragraph (iO) : 

[0009] A lec:ding syBten-. for cosn:etic surgery BA:::u].at ions is called the Mirror 
Suite. TM, Total Patient IrvtagiriC Software, A surgeon's C::::ice Vs?ith thn.s Bystem Ib 
able to have patients watch on screex). as they are guided frorvi their current 
appearance to a representative post operate ve Gut:coT::e (v;v;v; . can fieldsci , com} . 
H: mp 1 o y i ng mo r p h i ng and simulation tooIS; the Mirror Sudte.TM, l:i:aging systerr^ is 
applicable for resurfacing simulation, hair restoration tools, hair loss sin:ulation, 
and image management. Also provided are drav; and warp tools, and tools for airr-ulated 
breast augr-ientat ion . 

[odlQl The Canfield Mirror " Suite . TM > software systerr; can be incorporated vd.th the 
Canfield V.'erniaGraphix.TM. Dermatology T.rr^aging SG::tv;are system, v;hich otiiers 
dermatologists a cornpiete digital imaging systent utilized i:or serialiy tracking 
over time skin lesions located on the body of a patient. This systen: is used for 
pigmented lesions, and body mapping is also useful -lor documenting psoriasiB^ eczema 
and other conditions. Images purportedly can be cor-ipresBed. sufficiently to be sent 
via e-U'iail or posted on a v;eb page. 

Summary of invention Paragraph -12) : 

[OOllJ No description is wiade in the Can::ield website or brochures^ however, of a 
patient having the ability to automodulate his or her ov^?n appearance coi::pleteIy 
au'i-ornat ica!t ly ;::nd completely oxi-livxe via the Internet or other long- di;i;ta -iice 
carrier, \^rt:honi:. ax- i n-i.ervti^x] i i?.g t^;c:bn.i c^iar- or ot.her eu.pport: perS'ii-r-r-el , Jr- t:be 
Canfield system, then, it is not the patient , but the techniciex3 v/ho i;:; ir:^ charge of 
tYie sijt'Uii.at; ion ar-d morphing jiunctione. . . _ - - — 

.r.!i!!^:H£X lnventiox3 Paragraph -13); 

[0012 j Another corupany involved ia an i.magring and archiving i^iystetn for the plastic 
and cosmetic surgical arena is United lit-agirsg. Inc, v/hich cortimerc ia 1 1 y ofiiers it;H 
MarketWise .TM. Iraaging and Archiving System. Pea-:uring a high resolution camera. 
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patient photoqrapl:^:-; car; be taken { www . uirflaa:Lr•^^ . coff-) . A c;c:>Ior LCD o-; the caa-era cavx 
al ].C5V/ Jior j.:c3B*:artt rev:iew ov. images aad ;:<]'u:;v^tj caiiUirra sje-i-tirigtj Jioi' t;;=sc;b image - The 
color LCD on the di^ix^l caiwera i^vpically operates in Live ■■•ode the picture :t 

;:;napped. Voice files; aiay be savod oach picture . Tho Mar:k.etW;!.;i;e - TM , In^agiiig 

Sv/:^ten^^; t^avo ^;pec:iaiiy de£^:^gn^;d pji-ograiTts J:or s brail at i^:;r: o1: hai:-: replace^Tie-i:;., laser 
r^jsurJiaoing , and i.-pper- blepbar'cijplasty. Marke-:^! , TM . T-nagi/no ;incluc3ee advaxiced 
•:ool:^; (:cr cloni.no, morphing, ar;d eyniinet r: ioal morphing . Vol un-etr Ic morphing ::^imu j.a '^erf^ 
breae-: onla r:ceniont aad r^^-oxicii^on . A:n a-itomor:ph furK:t:iori al lov/e mox'phing w,;.t:bou.t 
firsit defii-ning a region. Using computer imaging as an in::egral part of the 
consultation process is a prov;en practice builder, adding increased patient 
understanding and the conf icience to proceed with proposed procedures? . 

^vummary of Invention Paragraph {17) : 

[0016] The Niamtu Ivnaging system soft's/are is alsJO ai;ailable { ^^/ww . nia^at-:; - con-) . The 
•<A/ebsite describes softv^are including the TIMS (Total Image Management) profeBSicnal 
image e d i t : i ng , p r e s e n t a t i on soft v;a re, s op h i s 1 1 c a t e d a r c h i v i no s o f t: a re , image 
capture software, global viewing softv;are, fast forrnatting. web toola, the ability 
to add pictures to referral letters, and others. Por orthodontic and cosmetic 
formatting. Nia^xtu Imaging Syst:eim is especially designed for Windows? 95 and 98 and 
Microsoft Office. Using this sysftein, the purchaser is able to perform a variety of 
tasks, including "pick and apply" editing which facilitates real-time ivviaging 
changes. Images can be dragged and dropped into referral letters? and other 
documents. The system has autoproces?3inq tools v;hich automatically straighten, crop, 
focxiSf and adjust for color and brightness v/ith the click of a mouse. It also has 
other special effects that allov; a surgeon to create Internet pages, and to 
catalogue any image from a patient to an art collecE:ion. Included is a tool for 
morphing . The system is useful for laser resurfacing, as well as being useful for 
orthodontic and cosmetic surgery f orvnatting . Web tools are also provided to create 
v^eb pages. The system is available on CD-ROM only, with an instructional video and 
manual. A digital camera is recommended for capturing digital images . 

Suwimary of Invention Paragraph (18) : 

[0017] A gection of the -Niamtu website entitled ''The Magic o;: Image Morphing,*- 
Btatee that the software wall automat:i.calIy redrav; changes '''ade by the user^ arid the 
color; shadow and cov-trast will be c^slculated automatically, usually withiin seconds. 
Moving the control points redraws the image in real time , Morphir^g can be done or- 
ar-y anatomical area -Iro::: breast to tooth. The user may al;so t^ooil: :^.n on a 3pecif:i.c 
body part, such as the nose, chin, etc., and isolate the structure which gaves the 
user wiore cotrtrol points and more precise changes without disturbing the rest of the 
^inatomy. "besides raorphing, the Niamtu Imaging Systeri: allo\^?s the user to rotate any 
anatow-y, including straightening a tooth or rotating cs. nasal tip. Text can also be 
added to any image . 

Summary of. Inverj t ion_Paragra^ (19) \ 

TooTSl The""N HuSu "v^ has a '-Presentation and Marketing" section, and an '-album 

•iunction,*' which allovs^s the user to place images in an albuii- in thumbnail format. 
Orice the images are saved to an album,- the user can make multimedia slide shov;s 
using the images . These slide shows may be mteriiaced with programs like Microso::t 
Power Point to make custom lectures and pro::essional publications. They can even be 
posted on the Internet. Additioual 1 y , the system provides for the developwient c-: 
custom databases, with which it is possible to categorize and retrieve images based 
upon specific search parameters, such as t;;eatmei?.t complications lirom a specific 
procedure, Sophisticated " Image Archiving" is available with the isTiamtu system, 
v;hich allows the ability to store images in over 40 formats, such as jpg^ bmp, tigg, 
gif, pcx and others. Compression is used to preserve hard drive memory. 

Summary of. i c>YrA*v.-^-9^1..-?:^£^^;^P^ '^^^ * 

To 0 1 5 J rb"«r^^^^ ToolB" i^iection of the website ir-dLcate;? that; in the 

future, imaging will rely more and more or- the Viorld Wide v^eb . A Sma r^rSaver tool is 
providech which aj. lows the uiiier t : p _ o o t j. ird e „ i ma g e s . --t ■ :> . wh.$^n., c: e 6. on .the^Tntemetf- 
they download quickly. The Niacntu sys;:.e:i: alsc? offers other special eKiiects, 

Sunvir;ary of Invent icr3 ^■■<^£^^apj' (23) : 

[0022} The computer imaging ::iection of thi:^ website is laterriet -ba-^ed, i: rom the 
p03n3t: of vi.ew n.hat: a per;;!^:5r; iii-:.erestBd. :ln ccAy^net±c plastic surgercy t:ran:?::iit:s a 
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Bcanned photo^ rapli on- Line d;;.r^ic-:ed to t:b.e 'nedical :Lllusi:rators on the s^aff ov. •^he 
Pl;^s-::ic SuTQitry Covr^pavi.y: ar:6 , afiiier i^he^-^ p^:of:essiox]ali>. inorph. -i-he o?:i9:irtal 
p^;ot09::^^:p^i:^^ the i^riiw pho-^o^^raphs depicting the po£^:^ibi ]. 1 1::^ e;ii ol: cc:>:^•net i.c; iitL-provevaent 
via -^he veh;.c!le of: ' pla:?t;i.c surgery o?:e trarisva:- i^t^iid oa-iline back t-o t:b.e prospecM:.ive 
candidat:^ tor i^x-roery. Ai: r;>::> ti.ir;^; however, do patients •:hen::^!e LveB have •:he 
oppo?: ;:.:iriity i:c> ohar-ge i:^-!^.}.;^ appear;n:c]ce or to exp^i-r 3.5r;e;ii:. v^;itb cho varrxo-i;:^ uT;ulat;i.-:;r; 
a lid morphing opt:iori;::. 

Summary of Invention Paragrciph {28) : 

[0026] The present invention of virbixal cosr^;etic auE:osurgery via teiernedicine 
constit:utes a new- v;ay of pert orrr-xng ^turqrcai siri-uiations and morphing . The patient 
it; ixi cooTpiete con^ir-and of the morphing and simaiation functions, and, interestingiy , 
such an individual is? not oniy the patient, but also becomes the surgeon? hence, the 
teriT;, -'autosurgery . " An image of the patient is provided to a central compuE:er. The 
patient xises a remoteiy located computer to connect to bhe central coi^u^:er, via a 
data transmission sysi^em, such as the Internet: . The patient can then input other 
patient data, as \:/eii as inputting desired modifications to the patient ' s own image, 
such as a face iift or breast augmentation. The patient can view tihe resuits of the 
rnodif ications on the revr-oteiy located computer. The present invention, in other 
v;ordSf provides the ability for the patient to be in control of directing the 
softv^are which controis the simulation process. The patient at a distant site, m 
essence, becomes the "surgeon" and directs the chosen procedure via interactive 
discourse with E:he computer sofEi^are system at the central processing site. 

Det:aii Description ^-aragraph (2) : 

[0 028] The present: invention operat:es in the foiiov>ing manner. A patient scans one 
or more photographs of t:he face, or other area of the body contemplated for virt:ual 
cosmetic autosurgery, into a home or office computer. Generally, frontai and profile 
views are required for optimai and realistic results. Alternatively, a photograph is 
mailed in to a provider of the simulation services, or a video camera attached to an 
office or home computer digitally scans the patient ' s face or body directly into the 
computer's memory, obviating the need for a photograph to be available. The content 
of the :3oftware selects appropriate views, v;ith the most useful being frontal and 
profile viev^s. Three dimensional modoling^ which requires 360 degree viev/s for best 
results, could also be used. As video camera attachmev^.ts become ubiquitous, this 
alternative is eixvisioned. as the future preferred embodiment. Until such tame as 
suitabl e -qual i ty video cameras for attachmerjt to computers become commonplace, 
however, pictures captured with either analog, or preferably, the newer digital 
cameras, "will suffice to provide the platform for cosmetic plastic autosurgical 
simulatiov^s and morphings . 

Detail Description iraragraph (3): 

[0029: Th-e d i g 1 1 .i z e d pic "t u res obtained as described above are trar:smitted ':rom the 
distant remote site, i.e., the home or o;:fice ov. the patient, to the company's 
central processing corriputer site, tor example via long- distav-ce carrier, pre -ler ably 
the Internet, and then appropriately calibrated and si-zied by the central site's 
system software, such that these v^ill be compatible ii- si^^e, position, and view 
angle tor the surgical procedural options available. For instance, if one's nose is 
to uT-dergo virtual cosmetic autosurgery, it is vital that the original photographs 
be proportioned and scaled m such a v^ay that the virtual surgical nose opt:i.ons can 
be superimposed properly onto the original photographs. This requires proper sizing 
of. the original photographs as the beginning first step. To elaborate, an individual 
may prefer, among other virtual surgical options, to have a -'smaller-- nose, a 
■■bigger-' nose, a more -'pointed'' nose, a --flatter-- nose, a "straighter" nose, a 
'■thiiu-.er nose; a nose without a "hump," or a nose with a -'hump,'" etc., but, in all 
of. these cases, it is imperative that the original photographs be digiti^red and then 
automa;:.ically sized and prcport.ior-ed in such a. ■■•anr-er that uniformity and 
standardization are achieved. 1:! t:housands of persons imi;.Itaneo\j.:?ly ^vish to perform 
rea 1 i stic v i r tual an tosurgery via t el emedic ine , such s tar-de rdi :?;at icxj i s manda tory 

_::.o?:: .the _ent.ix:e_n].e.': :Wor;c_t;o _f un(::t-ix>n__prop-e?::ly .i^rio. lor. p;:it ian;:.s__tf:;_x;.e_ able.__to _a:!!;:;.e.s.s?_ 

accurately the results of their telemedicixje virtuaj. autosurgery. It would most 
highly undesirable; for instar-ce, if a patient's new vLrtual autosurgical nose also 
covereci l-i;;; or: her mouth, due to irrxprooer i^izivx^ oi. the ori.ginal photograph. 

Detail Description Paragraph (13} ; 



u Record Display Form 



http://westbrs: 8002/biiv'gate.exe?f=doc&s. . .c_2=&p_doc_3=&p_doc_4=&p_doc_5=&p_do 



[0039] Cer::,ai;::}.y , i:-he p^:::;erit iviay wish t:o -i-ake pictures or a (: Loppy di^jc, or other 
:i;ui.t:ab!t-i: ^itora^}-;; inediut;-, o:: i^he fir- i.Bli^hd au tosMircfico 1 procedure t^:; a qiiaii (: 
plaj^itic or co^in-ie'::Lc j^iurgeo:;, but •:.here Is ■:::ert:aln.!ly no aeed or pre:?;:<ure placed to do 
iiio. Co::i;:5uit ;.o.o with a ?:'ea:i.- ). i J:o plasj:;:^:: or coBc-iotic »u3:'9^i-orj is ont i. r:e!l optxoaai, 
ba:Sod OX) ai; ir;d lv:Uiuai ' s w:t;:?hi;:s. Thi;:; dii:1:er:i; qi-i;:e :i!ub;;*:aritia ]. 1 y from o doctor's 
i;::-o;:J::ico corapator iT^od^^^iing oo:::?uito •::Lc:>ri;ii , v/b.:LCh aro ^^ypicali/ porf:orr(-ed ir; -^ho 
£;argo<:;r} ■ £5 c:>ff.Lc:3 by a Xiachv^iciax) . In these iiiituo -^li ort;:; , pav:ient^; oencriii 1 1 7 rnay hove 
either intecscSed or x- cr^ ntended pro;;:^;X're oxert:ed to h^ve one, or another; ;:n.:rgicai 
procedure . 

Detail DeBcription Paragraph (14/ : 

[0040] The Plastic Surgery Company's Btrategy, def?crrbed above, ±e "on-line,'' bat 
the surgery ie not autoperf ormed . Rather, trained technicians perforin the virtual 
surgery over a 24-hour period, and then transmit the final result back to the 
patient via the Internet. At no titr-e does the patient herself or hin^self have the 
on-line opportanit:y {or any opportunity) to self -operate - The final pictures are the 
by-product of the technician's ingenuity, personal biases, and viev^rs of the ideal, 
rather than, more democrat:ically, represent:ixig the E:rae v\?ishes and desires of the 
patient, as is the case -^'ith the present: invention. With the present invention, the 
patient herself or himself has the risk-free (and interesting} experience of 
perforir-ing autosurgery- all v?ithout physical risk and completely reversible with the 

push of a button, v^ithoat pressure to sign up for real -live surgery with real -live 

attendant risks - -and vjith extrevi:ely minimal expense, such as a nominal on-line 
charge. Even an individual of lirai*:ed means should be able to afford the present 
invention- s telemetric virtual cosmetic autosurgery. A vv-ost novel aspect of the 
present invention, then, centers around a person =s ability to interactively alter 
one's virtual appearance oneself via virtual telemedicine . 

Det^ail Description Paragraph (22) : 

[0047J 3. Person at Distant Remote Site responds to questions posed by Central 
Processing Site computer m Step 2. and then transmits scanned photocjraphs (or 
images frowi video camera) to Central Processing Site, which processes images into 
trie proper predetermined 3tc;ndardi'::ed format for ccmputer:i.";::ed niorphin.g and 
siuvulat ion . 

CIA IMS : 

1. A gel f "perf crrvied virtual cosmetic ;3urgery systeu:, comprising: a plurality of 
computers at a plurality of ren-ote sites, said plurality of revi-ote corviputers each 
having a video display device and. at least one input device; a cow-puter at a central 
site, sard central computer being prograinmed to receive original digital images of 
patients, to receive patient data, to receive patient image modification cou-inandB, 
to standardize said original patient images , to analyze said original patient 
images, to analyze said patient data, to analyse said patient image modification 
comm^^nds according to current medical protocois, and to generate modified digital 
patient images ; and a data transmission system connected to transmit said image 
modification commands from said pluraiity of remote computers to said, central 
con-puter, and to transmit said modified patient images from said central computer to 
said plurality of remote computers. 

8. The virtual cosmetic surgery eyatem recited in claim 1, wherein said central 
computer is further programmed to provide alternative image modification schemes to 
said plurality of remote computers, 

9. The virtual cosmetic surgery systervi recited in claim 8, v;h.erein said image 
modification scheme comprises a computerised scheme for modifying the size of a 
selected body part. 

10. The virtuail cosjmetio :Hurcery :^;y^;te^n recited in claim 8, v/herein eaid image 

r(\od±f.±cation scheme -cop-^pri S;::S_.a J^c>f^ip•iter:.i.^^ecln^ic:h . f •:;r rac^dif ;yit3o _::.he- -ori-eni:.at ion - of^ 

a iHclected body part:, 

12. The virt;ual co;i;metic :;iurce;:"y jiiystem reciteci in clai::- 8, v/herein said image 
modif ic.:: tion ijicbeme comp?:ises a computer i^^ed ;;ielec::.ion of alternative cool: ;igurat ior;s 
for a selected body part. 
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13. A i>.elt~perforvi^e6 virtual cor^i-net 3.c :?ur9er:y :^y::;te!T;, co-npr 1 r;^ ; p!; liral i. t:y of 
cc>[i:puter:B at a plurality oi: r^;nx;;:e i^iteii, i^aid pluraJ.ity of ren*;cce coii^pU';erB each, 
hiivina a vi.cJao d:Lspj.av/ device: a pat:ie:;t image input: device, arid at lea;i<t <;:c]e ciata 
iapui:: dev:U:'e; a c^on-pute?: a-:, a ceEi^rai ^Ji::e; sai.d ceritral ;::on:put:er beiriq progran-;iriBd 
t^:) >:Bc::eive c^Kilgina ). d:ioit:al. images o;: patie:c)-:£? via «a:id patlea-:. image ;triput: device, 
•:c> i:-ecei.ve pati.^en.':. data v:ia ;i;ai.c3 a-:, lean-: one dai^a input: cS.evic:e, ;:o receive pat:i.eii-:, 
image niodi i: ic^ation c;5^^•[i;and^> v:ia Bai.<3 a-:, leas*: c:>ne da-:a input: d^i^vice, -^o standardize 
said orioj-nal par:ient images, to analyze said original pai:ient images, t:o analyiie 
said patient: data, t:o analyi:e said patient image modification commandB; accordinc; to 
current medical protocols, and to generate modified digital patient images ; and a 
data traneiv-isBion system connected to transrifiit SJald original digital patient images, 
said patient dat:a, and said patient image modification commands from said plurality 
of remote computers to said central computer, and to transmit: said vr-odified patient 
images from said central computer to said plurality of remote computers. 

14. A mer:hod for self -performance of viriiual cosmetic surgery, comprising: providing 
a plurality of remote comput:ers at: a plurality of remotely located sites; providing 
a central comput:er at a central site; inputting an original digital image of a 
patient into said central computer; inputting data relative to said patient into 
said cen^:ral computer; inputting image modification covc\\C:avids int:o said at least one 
remote computer; transmitting said image modification commands from said at least 
one remote computer to said central computer via a data transmission system; 
standardizing said original patient image \vith said central computer ? anaiyiiing said 
original patient image, said pat:ient dat:a, and said image modification commands, 
according to current medical protocols, v;ith said central computer; generating a 
modified patient image Vi/ith said central computer, according to said image 
modification commands; transmitting said modified patient image from said central 
computer to said at leas?: one remote computer via a data transmission system; and 
displaying said modified patient image on a video display device connected t:o said 
at least one remote computer, 

15. The virtual cosmetic surgery method recited m claim 14, v;herein said 
standardising oi: said original patient image comp7:i3e3 revis.irig said image to a 
:5ta^3da.rd size to match image siv?:v:s ::or vario^js a].ter:i;uJ.tive body part conxiigurcS-ti.oriS . 



16. The virtual cosmotic surgery method recited in claim X^, v^herein said analysis 
of said original patient image, eaid patient data, cuid sard image modification, 
commands; comprises calculation of expected outcomes for image modification schemes 
selected by a particular patient. 

17. The virtual cosmetic surgery method recited m claim 14, further comprising 
providing alternative image modification schemes to said plurality of remote 
coiT^uters, v;ith isaid central computer. 

18. The virtuai cosmetic surgery method recited in claim 17, ^vherein said provision 
of alternative image modi f i cat! ot- schemes comprises providing a computerized scheme 
for modifying the size of a sei acted body part. 

19. The virtual cosmetic surgery method recited m claim 17, -wherein said provision 
of alternative image modification schemes comprises providing a computerized scheme 
for modifying the orientation of a selected body part . 

21. The virtuai cosmetic surgery method recited m claim 17, v;herein said provision 
of alter\?.ative image modification, schemes comprises providing a con-puterized 
litelection of alt terr-e ti ve cct3 f igurat i-ons for a selected body part. 

22. The virt\3a!t cosmetic surgery method recited in clain: 14, further coinpr i.si ng 
inputting said originat digi tal image o:: a patient -into -sad d- ceT3tr<j.:t co'Tipui:.er via -■ - - 
:;!afd at least one remote coir^puter end vi.a said date tran::im3.ssion system. 

23- The virtuai c^;smetic surgery method recited in claim 14, further coinpr i.s i ng i 
provid3.x3g a photograph of a patient to en operator of said central computer vie a 
mail service: and inputtir-Q said original digital image of the patient into said 
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ceatriii]. coEiiputer: via ar; image i.:c3;pi:i,;:. device c^:o::ri^*c;;ed to £5a;td cen;:ral cOT;pu-:.-::r . 



